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Agenda 



* Risk Management (RM) Recap 

- What is Continuous Risk Management (CRM) 

- What is Risk Informed Decision Making (RIDM) 

What is Risk Management (RM) 

* What does PR A mean in the context of RIDM 
4 NASA’s newest Pl^A requirements 

* How does GSFC % flow down PRA requirements 

- Two Approaches v 

• In-House PRA Development 

* Out-of-House PRA Development 

- GSFC’s Standard Mission Assurance Requirements (MAR) Document 

* Discussion 
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What is RIDM? 



Step 1 - Understand Stakeholder Expectations, 


Step 3 - Set the Framework and Choose the 
Analysis Methodologies 
Step 4 - Conduct Itte Risk Analysts and 
Document the Resuds 


: ; ?^6imianc.e 


Step 6 - Deliberate Select an Alternative and 
Document the Decision Raliofiale 
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http://www.hq.nasa.gov/office/codeq/doctree/SP2010576.htm 
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Continuous Risk Management (CRM) 



Pfogram/project data 
(metrics information) 


Note: Communication and 
documentation extend 
throughout all of the functions 
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What is Risk Management? 











What is PRA? 



Definition : x^x 

Probabilistic Risk Assessment (PRA) is one of key RIDM tools. It 
a scenario-based methodology aimed at identifying and assessing 
Safety and Technical Performance risks in complex technological 

systems. 

PRA characterizes: 

1 ) What can go wrong? 

2 ) How likely is it? 

3 ) What are the consequences 

Triplet Concept: Scenario, Likelihood, Impact 
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What is PRA? 






What is PRA? 

















What is PRA? 



The integrated 
risk scenario 
model allows for 
a robust and 
ordered ranking 
of the risk 
contributors that 
may lead to the 
undesired states 
of interest 
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Two-Sets of Requirements governing 
PRA activity at GSFC 



Applicable requirements pulled from: 

* NPR 87Q5.5A PRA Procedures for NASA 
Programs and Projects (2010) 

* NPR 8715.3C NASA General Safety Program 
Requirements (2012) 
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Some of the Significant Changes to NPR 
8705.5 made in conjunction with RIDM 



Removed language addressing Full, Simplified, and Limited 


Scope PRA. 

• Greater emphasis on PRA Planning/PRA Plan Documentation 

- Formal PRA Plan shall be developed by PRA Lead and approved by 
Program/Project Manager to: 

-- Identify specific end-states (undesirable consequences, performance 
measures, figures of merit) of interest consistent with PRA objectives 

- Define quantitative performance measures and numerical criteria that are to 
be evaluated by the PRA consistent with objectives 

- Develop a PRA schedule compatible with objectives, applications, and life 
cycle phases approved by the Program/Project manager 

- Greater tie-in with Program/Project characterization of 
NPR 7 1 20.5 NASA Space Flight Program and Project 
Management Requirements 

- Use of Category 1 (i.e., LCC greater than 1 Billion dollars, 
Nuclear Payloads, Human Spaceflight) 
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New NPR 8705.5 Requirements for GSFC 



- Formal submittal of PRA Decisions - Cat I and Cat II, 
Class A or Class B. 

-- Formal submittal of PRA plans, including scope and 

rationale 

- Requires Program Manager approval (note: PRA 
decision/scope determination is the primary 
responsibility of the Project subject to OSMA/Center 
SMA Concurrence) 

- Formal coordination of Independent PRA Review with 

OSMA. 
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How is GSFC Flowing Down PRA 
Requirements to Developers/Suppliers 



• GSFC’s Mission Assurance Requirements (MAR) 


PROBA BI LISTIC RISK ANALYSIS (PRA) AND RELIABILITY 

Tailoring note: The PRA and reliability engineering section requires tailoring per the classification requirements o/NPR 
8705.4, NPR 8705.5, and project-specific requirements. 


RELIABILITY PROGRAM PLAN 

Tailoring note: If PRA is being invoked in section 4.2, change section 4. lto read from “...implement a Reliability Program Plan 
(RPP)” to "... ”, implement a Reliability Program Plan, including the developer's approach to PRA requirements in section 4.2, .. ” 


The developer shall document and implement a Reliability Program Plan (RPP) using both qualitative and quantitative techniques 
to support decisions regarding mission success and safety throughout system development. The RPP shall include a detailed 
approach to the analysis of hardware and software for their contributions to system reliability and mission success. The developer 
shall present the implementation of these plans and related activities at milestone reviews beginning with the System Requirements 
Review (DID 4-1). 


PROBABI L ISTIC RISK ASSESSMENT 

Tailoring notes: See paragraph 2.2. la of NPR 8705.5 for criteria regarding the requirement to perform a PRA. If a PRA is not 
required, delete this section and the related DIDs. If a PRA will be performed, delete the non-applicable paragraph and related 

DID. 


The developer shall perform a Probabilistic Risk Assessment in accordance with NPR 8705.5, Probabilistic Risk Assessment (PRA) 
Technical Procedures for Safety and Mission Success for NASA Programs and Projects (DID 4-2). 


The developer shall provide the information for a Probabilistic Risk Assessment per NPR. 8705.5, Probabilistic Risk Assessment 
(PRA) Technical Procedures for Safety and. Mission Success for NASA Programs and Projects (DID 4-2). 
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How is GSFC Flowing Down PRA 
Requirements to Developers/Suppliers 



• MAR Data Items Description (DID 4- 1 ) 


Title: Reliability Program Plan DID No. : 4-1 

MAR Paragraph: 4. 1 
Use: 

Planning and implementation of Probabilistic Risk Assessment (PRA) and reliability activities. 

Reference Documents: 

NPD 8720. 1 , NASA Reliability and Maintainability (R&M) Program Policy 

NASA-STD-8729. 1, Planning, Developing and Managing an Effective Reliability and Maintainability (R&M) Program. 

NPR 8705.4 Risk Classification for NASA Payloads 

NPR 8705.5 PRA Procedures for NASA Programs and Projects 


Place/Time/Purpose of Delivery: 

Deliver draft plan to the Project Office sixty (60) days after contract award for review. 

Deliver final plan to the Project Office thirty (30) days prior to the Systems Requirements Review for approval. 

Deliver activity reports related to implementation of the plan at milestone reviews beginning with the Systems Requirements 
Review for review. 


Preparation Information : 

The Reliability Program Plan shall include: 

A discussion of how the developer intends to implement and comply with Reliability program requirements. 

Charts and statements describing organizational responsibilities and functions conducting each task to be performed as part of 
the Program. 

A summary (matrix or other brief form) that indicates for each requirement, the organization responsible for implementing 
and generating the necessary documents. 

Identify the approval, oversight, or review authority for each task. 

Narrative descriptions, time or milestone schedules, and supporting documents describing the execution and management 
plan for each task. 

Documentation, methods, procedures, and reporting specific to each task in the plan. 
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How is GSFC Flowing Down PRA 
Requirements to Developers/Suppliers 



M AR DID 4- 1 - use this DID if the developer is performing the 


PRA 


October 19, 2011 


T itle: Probabilistic Risk Assessment 1 D ID No .: 4-2 

MAR Paragraph: 4,2 ’ 

Use: 

To provide a structured and disciplined approach to: analyzing system risk: supporting management 
decisions; improving safety, operations, performing maintenance and upgrades; improving performance: 

reducing costs. 

Reference Documents: 

NPR 8705.4 Risk Classification for NASA Payloads 

NPR 8705.5 Technical Probabilistic Risk Assessment (PR A) Procedures for Safety and Mission 

Success for NASA Programs and Projects 

NPR 87 15.3 NASA General Safety Program Requirements 

PRA Procedures Guide for NASA Managers and Practitioners 

(http://www.hq.nasa.gov/oflice/codeq/doctree/praguide.pdf) 

Place/Time/Purpose of Delivery: 

Deliver a PR A plan to the Project office sixty (60) days alter contract award for review (Note: PRA 
may he stand-alone document or included as part of the Reliability Program Plan (RPP), Risk 
Management Plan (RMP), etc. The PRA Plan shall meet requirements delineated in DID 4-1 .). 
Deliver interim PRA to the Project Office thirty (30) days prior to PDR for review. 

I teliver updated interim PR A to the Project Office thirty (30) days prior to CDR for review. 

Deliver updated interim PR A to the Project Office thirty (30) days prior to MOR for review. 

Deliver final PRA to the Project Office thirty (30) days prior to FOR for approval. 


Preparation Information: 

The PR A shall be performed in accordance with NPR 8705.5 and include the following: 

The objective and scope of the PR A 
End-statcs-of-interest to the decision-maker. 

Definition of the mission phases and success criteria. 

Initiating event categories, 

Top level scenarios. 

Initiating and pivotal event models (e.g., fault trees and phenomenological event models), including 
assessments of common cause failure modes 
Data development for probability calculations, 

Integrated model and quantification to obtain risk estimates. 

Assessment of uncertainties, 

Summary of results and conclusions, including a ranking of the lead contributors to risk. 
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How is GSFC Flowing Down PRA 
Requirements to Developers/Suppliers 



MAR DID 4-2 - use this DID if the GSFC is performing the PRA 


Title: Information for the Probabilistic Risk Assessment (PRA) l 

DID No.: 4-2 

MAR Paragraph: 4-2 



Use: 

To provide a structured and disciplined approach to: analyzing system risk; supporting management decisions; address safety, operations, 
maintenance, and upgrades; manage performance; manage costs. 


Reference: 

NPR 8705.4 Risk Classification for NASA Payloads 

NPR 8705.5 Technical Probabilistic Risk Assessment (PRA) Procedures for Safety and Mission Success for NASA Programs and Projects 
NPR 8715.3 NASA General Safety Program Requirements 

PRA Procedures Guide for NASA Managers and Practitioners (http://www.liq.nasa.gov/office/codeq/doctree/praguide.pdt) 


Related Documents 
None 

Place/Time/Purpose of Delivery: 

Deliver preliminary heritage information, including the percent applicable, to the Project Office sixty (60) days after contract award for 
information. 

Deliver updated heritage information, including the percent applicable heritage to the subject mission, to the Project Office thirty (30) days 
to prior major milestone reviews beginning with the SRR for information. 

Deliver product information and process information for elements within the scope of the Mission PR A to the Project Office thirty (90) days 
prior to the PDR and thirty (30) days prior to subsequent major milestone reviews for information. 


Preparation Information: 

The government will provide a notification to the developer of the scope and/or area of inputs needed to support the risk assessment 30 days 
prior to needing information in preparation of the PRA. Types of information needed may include heritage information (eg., current flight 
history, current operating hours, operational and storage environments, TRLs, etc,), product information (e.g., hardware and/or software 
configurations, parts lists, schematics), interim analysis (e.g, working-level copies of fault tree analysis, failure modes and effects analysis, 
reliability predictions, etc) and/or process information (e.g., design documents, manufacturing documents, parts program documents.etc) 
germane to the elements) being evaluated within the scope of Mission PRA and instrument development. The developer and their 
collaborators will provide access to the information necessary to support the scope of the Mission PRA. 
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Lessons Learned 



• It is important that a clear scope, set of objectives, and 
milestones be established upfront with the project team 
to help ensure that needed questions will be answered 
in a timely manner. 

• PRA and other models rely heavily on the degree and 
fidelity to which they match what will happen in 
reality. Such modeling requires close coordination 
with the design team, and a sound system 
understanding from the modeler. 

• Need to establish a framework for developing basic 
event likelihood distributions upfront. 

• Need to clearly communicate results 
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Discussion 




* Comments 

* Questions 
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